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SYSTEMS AND METHODS FOR DETERMINING 
DEFECT CHARACTERISTICS OF A COMPOSITE STRUCTURE 

COPYRIGHT NOTICE 
5 [0001] A portion of the disclosure of this document contains material 

that is subject to copyright protection. The copyright owner has no objection to 
the facsimile reproduction by anyone of the patent disclosure, as it appears in the 
U. S. Patent and Trademark Office patent files or records, but otherwise the 
copyright owner reserves all copyright rights whatsoever. 

10 

FIELD 

[0002] The present invention relates generally to the fabrication of 
composite structures with material placement machines, and more particularly 
(but not exclusively) to systems and methods for determining defect 
15 characteristics of a composite structure, such as defect density-per-unit area 
and/or cumulative defect width-per-unit area. 

BACKGROUND 

[0003] Composite structures have been known in the art for many 
20 years. Although composite structures can be formed in many different manners, 
one advantageous technique for forming composite structures is a fiber 
placement or automated collation process. According to conventional automated 
collation techniques, one or more ribbons of composite material (also known as 
composite strands or tows) are laid down on a substrate with a material 
25 placement machine. The substrate may be a tool or mandrel, but, more 
conventionally, is formed of one or more underlying layers of composite material 
that have been previously laid down and compacted. 

[0004] Conventional fiber placement processes utilize a heat source 
to assist in compaction of the plies of composite material at a localized nip point. 
30 In particular, the ribbon or tow of composite material and the underlying substrate 
are heated at the nip point to increase the tack of the resin of the plies while 
being subjected to compressive forces to ensure adhesion to the substrate. To 
complete the part, additional strips of composite material can be applied in a 
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side-by-side manner to form layers and can be subjected to localized heat and 
pressure during the consolidation process. 

[0005] Unfortunately, defects can occur during the placement of the 
composite strips onto the underlying composite structure. Such defects can 
5 include tow gaps, overlaps, dropped tows, puckers (i.e., raised regions in a tow), 
and twists. In addition, there are foreign objects and debris (FOD), such as resin 
balls and fuzz balls, that can accumulate on a surface of the composite structure 
which must be detected, identified and eventually removed from the ply surface. 
[0006] Composite structures fabricated by automated material 

10 placement methods typically have specific maximum allowable size requirements 
for each flaw, with these requirements being established by the production 
program. Production programs also typically set well-defined accept/reject criteria 
for maximum allowable number of (i.e., density) of defects-per-unit area and 
maximum allowable cumulative defect width-per-unit area. 

15 [0007] To ensure that the composite laminates fabricated by fiber 

placement processes satisfy the requirements pertaining to defect size, the 
structures are typically subjected to a 100% ply-by-ply visual inspection. These 
inspections are traditionally performed manually during which time the fiber 
placement machine is stopped and the process of laying materials halted until the 

20 inspection and subsequent repairs, if any, are completed. In the meantime, the 
fabrication process has been disadvantageously slowed by the manual inspection 
process and machine downtime associated therewith. 

[0008] Recently, systems have been developed that are capable of 
detecting, measuring, and marking individual defects in the composite structure. 

25 Exemplary systems and methods capable of accurately and reliably detecting, 
measuring and/or marking defects in a composite structure are disclosed in U.S. 
Patent Application No. 09/819,922, filed March 28, 2001, entitled "System and 
Method for Identifying Defects in a Composite Structure"; U.S. Patent Application 
No. 10/217,805, filed August 13, 2002, entitled "System for Identifying Defects in 

30 a Composite Structure"; and U.S. Patent Application No. 10/628,691, filed July 
28, 2003, entitled "Systems and Methods for Identifying Foreign Objects and 
Debris (FOD) and Defects During Fabrication of a Composite Structure." The 
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entire disclosures of U.S. Patent Application Nos. 09/819,922, 10/217,805, and 
10/628,691 are each incorporated herein by reference as if fully set forth herein. 

[0009] Although these inspection systems have worked well for their 
intended purposes, the inventors hereof have recognized that it would be even 
5 more beneficial to provide systems and methods that are capable of determining 
a defect characteristic of a composite structure, such as the composite structure's 
defect density-per-unit area and/or cumulative defect width-per-unit area. 

SUMMARY 

10 [0010] Systems and methods for determining a defect characteristic 

of a composite structure, such as defect density-per-unit area and/or cumulative 
defect width-per-unit area. In one preferred embodiment, a method for 
determining a defect characteristic of a composite structure generally includes: 
determining a first distance from a first reference point of the composite structure 

15 to a defect; determining a second distance from a second reference point of the 
composite structure to the defect; using the first and second distances to 
establish a reference area of the composite structure; and considering each 
defect detected within the reference area and producing therefrom a defect 
characteristic representative of the composite structure. 

20 [0011] The features, functions, and advantages can be achieved 

independently in various embodiments of the present inventions or may be 
combined in yet other embodiments. 

BRIEF DESCRIPTION OF THE DRAWINGS 
25 [0012] The present invention will become more fully understood 

from the detailed description and the accompanying drawings, wherein: 

[0013] FIG. 1 is a schematic view of an exemplary composite 

structure illustrating linear and lateral distances to a defect in the composite 

structure according to one embodiment of the invention; 
30 [0014] FIG. 2 is a perspective view of a compaction roller having a 

code ring coupled thereto for common rotation therewith and a photo sensor 

positioned to monitor the code ring according to one embodiment of the 

invention; 



3 



B ing R f. 03-0585 
HDP R f. 7784-000623 

[0015] FIG. 3 is a schematic view of the code ring shown in FIG. 2; 

[0016] FIG. 4 is a schematic view of a system according to one 
embodiment of the present invention; 

[0017] FIG. 5 is a perspective view of a system according to another 
5 embodiment of the present invention; 

[0018] FIG. 6 is a perspective view of a light source according to the 
system embodiment shown in FIG. 5; 

[0019] FIG. 7 is a perspective view of a system according to another 
embodiment of the present invention; 
10 [0020] FIG. 8 is a perspective view of a light source according to the 

system embodiment shown in FIG. 7; 

[0021] FIG. 9 is an video frame capturing a pucker and a twist in a 
composite structure; 

[0022] FIG. 10 is a view of a computer display and user controls 

1 5 according to one embodiment of the present invention; 

[0023] FIG. 11 is a view of an exemplary part model which may be 
imported from external or third party software according to one embodiment of 
the present invention; 

[0024] FIG. 12 is a view of the part model shown in FIG. 1 1 with a 

20 course grid overlay according to one embodiment of the invention; 

[0025] FIG. 13 is a view of the part model shown in FIG. 12 but with 

the course grid overlay repositioned to represent a change in orientation for the 
new ply according to one embodiment of the invention; 

[0026] FIG. 14 is a view of two computer displays simultaneously 

25 displaying the computer display shown in FIG. 10 and the part model and course 
grid overlay shown in FIG. 13 according to one embodiment of the present 
invention; 

[0027] FIG. 15 is a view of a computer display according to one 

embodiment of the present invention; and 
30 [0028] FIG. 16 is a view of a computer display according to one 

embodiment of the invention. 

[0029] Corresponding reference characters indicate corresponding 
features throughout the several views of the drawings. 
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DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS 
[0030] According to one aspect, the invention provides a method for 
determining a defect characteristic of a composite structure, such as defect 
5 density-per-unit area and/or cumulative defect width-per-unit area. In one 
embodiment, the method generally includes: determining a first distance from a 
first reference point of the composite structure to a defect; determining a second 
distance from a second reference point of the composite structure to the defect; 
using the first and second distances to establish a reference area of the 

10 composite structure; and considering each defect detected within the reference 
area and producing therefrom a defect characteristic representative of the 
composite structure. 

[0031] Preferred embodiments of the invention provide methods for 

determining for a reference area or region of a composite structure one or more 

15 of the following defect characteristics: a total defect count, total defect width, 
defect density-per-unit area (i.e., number of defects-per-unit area), cumulative 
defect width-per-unit area and/or defect location. Various embodiments allow 
these defect characteristics to be determined as the composite structure is being 
fabricated, thereby eliminating the need for manual inspection processes and the 

20 machine downtime associated therewith. 

[0032] In one embodiment, the method generally includes 

determining a linear distance to a defect along a course being laid by a material 
placement machine; determining a lateral distance to the defect from a first end 
of the composite structure; using the linear and lateral distances to establish a 

25 reference area; totaling defects within the reference area; dividing the defect total 
by the reference area to determine a defect density-per-unit area; determining a 
width for each defect within the reference area; totaling the widths of the defects 
within the reference area; and dividing the width total by the reference area to 
determine a cumulative defect width-per-unit area. 

30 [0033] In the exemplary embodiment, the method includes 

determining both defect density-per-unit area and cumulative defect width-per- 
unit area. Alternatively, other embodiments can include determining any one or 
combination of total defect count, total defect width, defect density-per-unit area, 
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cumulative defect width-per-unit area and/or defect locations. Further 
embodiments can include determining any one or combination of total defect 
count, total defect width, defect density, cumulative defect width and/or defect 
locations for the entire composite structure in which case a reference area need 
5 not necessarily be established. 

[0034] FIG. 1 illustrates an exemplary composite structure 22, which 

is generally comprised of a plurality of adjacent tows or strips of composite tape 
24. The strips 24 typically include a plurality of fibers embedded in a resin or 
other material that becomes tacky or flowable upon the application of heat. The 

10 strips 24 are arranged on a work surface, such as a table, mandrel, or other tool 
26 (FIG. 4), and compacted with a compaction roller 20 (FIGS. 2 and 5) to form 
the composite structure 22 according to an automated collation technique, such 
as that described in U.S. Patent Application No. 10/068,735, filed on February 6, 
2002, entitled "Composite Material Collation Machine and Associated Method for 

15 High Rate Collation of Composite Materials". The contents of U.S. Patent 
Application No. 10/068,735 is incorporated herein by reference in its entirety as if 
fully set forth herein. 

[0035] As shown in FIG. 1, eighteen courses or strips 24 have been 
completed by the material placement machine. That is, the material placement 

20 machine has made eighteen passes across a substrate. During each of the 
passes, the material placement machine has laid down a strip 24 on the 
substrate. 

[0036] With further reference to FIG. 1, the sixth course 23 of the 
composite structure 22 includes a defect 36 in the form of a tow gap. Additionally, 
25 or alternatively, the composite structure 22 can also include other types of 
defects such as overlaps, dropped tows, puckers, twists, and foreign objects and 
debris (FOD) with such defects being counted and measured by embodiments of 
the invention. 

[0037] The dashed line 19 represents the linear distance along the 
30 sixth course 23 to the defect 36. The dashed line 21 represents the lateral 
distance to the defect 36 from a first end 1 1 of the composite structure 22. 

[0038] Various methods may be used to determine linear distances 
along a course to a defect detected in that course. In an exemplary embodiment, 
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linear distance to a defect along a course can be determined by multiplying the 
linear velocity of the material placement head unit along the course with the 
amount of time that has lapsed between when the course began and when the 
defect is detected. 

5 [0039] When a defect is detected, a signal can be produced that not 

only indicates defect detection but may also trigger measurement and marking of 
the defect. Exemplary systems and methods capable of detecting defects in a 
composite structure are described generally below and in more detail in U.S. 
Patent Application No. 09/819,922, filed March 28, 2001, entitled "System and 

10 Method for Identifying Defects in a Composite Structure"; U.S. Patent Application 
No. 10/217,805, filed August 13, 2002, entitled "System for Identifying Defects in 
a Composite Structure"; and U.S. Patent Application No. 10/628,691, filed July 
28, 2003, entitled "Systems and Methods for Identifying Foreign Objects and 
Debris (FOD) and Defects During Fabrication of a Composite Structure." The 

15 entire disclosures of U.S. Patent Application Nos. 09/819,922, 10/217,805, and 
10/628,691 are each incorporated herein by reference as if fully set forth herein. 

[0040] The start and stop of a course can be determined using 
signals from the machine load cell which indicate whether or not pressure is 
being applied to the compaction roller 20 (FIGS. 2 and 5). Receipt of a "pressure 

20 on" signal from the machine load cell indicates that the compaction roller 20 is in 
contact with the composite structure 22 and therefore, that a course has been 
started. Receipt of a "pressure off" signal indicates that the compaction roller 20 
is no longer in contact with the composite structure 22, and therefore that a 
course has been completed. Accordingly, the time between course start and 

25 defect detection can be determined by tracking the amount of time elapsing 
between receipt of the "pressure on" signal from the machine load cell and the 
receipt of the signal indicating detection of a defect. 

[0041] Alternatively, course start and stop can be determined by 
receipt of a signal from a device employing proximity sensors, lasers, or sound 

30 detectors positioned for determining whether or not the compaction roller 20 is in 
contact with the composite structure 22. 

[0042] In one embodiment, the linear velocity of the head unit is 
determined by determining the angular velocity of the compaction roller 20 and 
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multiplying the angular velocity by a circumference of the compaction roller 20. 
Alternatively, other methods can also be used to determine the linear velocity of 
the head unit, such as by using a radar gun commonly used for law enforcement 
purposes in monitoring vehicular speeds along roadways. 
5 [0043] Referring to FIGS. 2, 3 and 5, the angular velocity of the 

compaction roller 20 can be determined by a code ring 1 coupled for common 
rotation with the compaction roller 20. As shown, the code ring 1 includes 
alternating contrasting portions 2 and 3, such as alternating black and white 
segments. In FIG. 3, the code ring 1 includes an outer diameter 4 of about 1.010 

10 inches and an inner diameter 5 of about .844 inches, although other ring sizes 
can also be employed. In other embodiments, the contrasting portions can be 
provided directly on the compaction roller 20 (e.g., marked on, painted on, etc.), 
thereby eliminating the need for the separate code ring 1 . 

[0044] With further reference to FIGS. 2 and 5, a photo sensor 7 

15 (e.g., an off-the-shelf photo diode, etc.) is positioned to monitor and capture real- 
time images of the light-to-dark transitions of the code ring 1 as the code ring 1 
rotates along with the compaction roller 20. By detecting and counting the light- 
to-dark transitions of the ring 1 , the compaction roller revolutions can be counted 
and monitored. The frequency at which the light-to-dark transitions occur can be 

20 used to establish the angular velocity of the compaction roller 20. Preferably, 
axial motion in the compaction roller 20 is minimized in order to maintain the 
distance from the photo sensor 7 to the code ring 1 constant, which, in turn, 
allows for more accurate determination of the machine head unit's linear velocity. 
[0045] In another exemplary embodiment, the linear distance to a 

25 defect along a course can be determined by counting the number (whole and 
fractional) of revolutions the compaction roller 20 makes from the start of the 
course to the defect and multiplying that number of revolutions by the 
circumference of the compaction roller 20. By way of example, the photo sensor 
7 and code ring 1 can be used to count the number of revolutions of the 

30 compaction roller 20 between receipt of the "pressure on" signal from the 
machine load cell and receipt of the signal indicating that a defect has been 
detected. 
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[0046] Various methods can also be employed to determine the 
lateral distances to defects from the first end 1 1 of the composite structure 22. 
See FIG. 1. In one exemplary embodiment, lateral distance to a defect can be 
calculated by counting the total number of completed courses, not including the 
5 course in which the defect resides, and then multiplying the average width of a 
course by the number of completed courses. This method is particularly effective 
for tape placement in which each course is the same width, i.e., the width of the 
tape. 

[0047] The total number of completed courses can be determined 

10 by tracking or counting receipt of the pressure on/off signals from the machine 
load cell. Receipt of a "pressure on" signal from the machine load cell indicates 
that the compaction roller 20 is in contact with the composite structure 22 and 
has thus started a course. Receipt of a "pressure off" signal indicates that the 
compaction roller 20 is no longer in contact with the composite structure 22 and 

1 5 has thus completed the course. 

[0048] For fiber placement courses in which the width of each 
course may not be equal, the lateral distances to defects can be accurately 
determined by employing a "software ruler." More specifically, the lateral distance 
can be determined by acquiring a digital image of at least the portion of the 

20 composite structure including the lateral distance; selecting a pixel set from the 
digital image that represents the lateral distance; counting the number of pixels 
comprising the pixel set; and correlating the pixel count with correlation data 
(e.g., a predetermined relationship between pixel count and distance) to compute 
an indirect quantitative measurement for the lateral distance. 

25 [0049] The width of a defect can be determined in a similar manner. 

After a digital image of the defect has been acquired, a pixel set is selected from 
the digital image that represents the width of the defect. The pixels comprising 
the pixel set are counted, and the pixel count is then correlated with correlation 
data (e.g., a predetermined relationship between pixel count and distance) to 

30 compute an indirect quantitative measurement for the defect width. 

[0050] Alternatively, defect width may be determined by multiplying 
the linear velocity of the head unit (as determined in a manner described above) 
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by the amount of time required for the head unit to traverse the distance 
separating the opposed sides of the defect. 

[0051] The reference area can be defined as any region of the 
composite structure which is currently under inspection for defects and which has 
5 a surface area about equal to the surface area of the reference area. Further, the 
reference area can be sized to include any suitable surface area, such as five 
square inches, one square foot, etc. In addition, the reference area can be sized 
in accordance with production requirements to include only a portion of the 
composite structure. Alternatively, other embodiments can utilize a reference 

1 0 area corresponding in size to the entire composite structure. 

[0052] The reference area can be established in various ways. In 
one exemplary embodiment, the reference area comprises any region of the 
composite structure that is bounded by the linear and lateral distances to the 
presently detected defect. For example, and referring to FIG. 1, a reference area 

15 can be established for the defect 36 as the rectangular portion of the composite 
structure 22 defined by the dashed lines 19 and 21 and the composite structure's 
first end 1 1 and lower side edge. 

[0053] In another embodiment, the reference area comprises any 
region of the composite structure which is bounded by a predetermined linear 

20 distance and a predetermined lateral distance. 

[0054] In either of the aforementioned embodiments, the bounded 
reference areas can be tracked during the inspection, for example, in a lookup 
table. The lookout table might then be compared to a running tally of defects 
(e.g., running defect quantity and/or running defect width) during the inspection. 

25 [0055] In yet other embodiments, the reference area is defined as 

the region of the composite structure which includes the preceding portion of the 
course in which the presently detected defect resides and all of the completed, 
preceding courses. For example, and referring to FIG. 1 , a reference area can be 
established for the defect 36 as the first five courses to the left of course 23 and 

30 that portion of the sixth course 23 below the defect 36. 

[0056] In further embodiments, the reference area is defined as a 
region of the composite structure which includes the preceding portion of the 
course in which the presently detected defect resides and a predetermined 
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number of completed courses immediately preceding the course in which the 
presently detected defect resides. For example, a reference area can be 
established for the defect 36 as that portion of the sixth course 23 below the 
defect 36 and the three courses (i.e., third, fourth and fifth courses) to the 
5 immediate left of the sixth course 23 in FIG. 1 . 

[0057] In certain embodiments, a comparison is made between the 

cumulative defect width-per-unit area and a maximum allowable cumulative 
defect width-per-unit area to determine whether a composite structure is 
acceptable or should be rejected. The maximum allowable cumulative defect 
10 width-per-unit area can be set by production requirements. When the cumulative 
defect width-per-unit area exceeds the maximum allowable cumulative defect 
width-per-unit area, the manufacturing process can be halted and/or an indicator 
of unacceptability can be provided, for example, by a user interface 76 (FIG. 4) 
described below. 

15 [0058] Additionally, or alternatively, certain embodiments include 

comparing the defect density-per-unit area and a maximum allowable defect 
density-per-unit area to determine whether a composite structure is acceptable or 
not. The maximum allowable defect density-per-unit area can be set by the 
production requirements. When the defect density-per-unit area exceeds the 

20 maximum allowable defect density-per-unit area, the manufacturing process may 
be halted and/or an indicator of unacceptability may be provided, for example, via 
the user interface 76 (FIG. 4) described below. 

[0059] An exemplary system 10 which can be used to detect 
defects in a composite structure is illustrated in FIG. 4. As shown in FIG. 4, the 

25 system 10 includes at least one camera 12 and at least one light source 14. The 
camera 12 is connected to a processor 66 for interpreting the images the camera 
12 captures, or to a storage device 64 for storing the images, or both, as 
discussed more fully below. 

[0060] The light source 14 is positioned to emit light for illuminating 

30 the composite structure 22. The illumination is reflected differently by defects in 
the composite structure than from portions of the composite structure that are 
defect free. For example, illumination reflecting off non-defective portions of the 
composite structure 22, and light that fails to reflect off of defects in the 
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composite structure 22, or vice versa, creates visible images that can be 
captured by the camera 12. Details regarding systems and methods for 
identifying defects in a composite structure during fabrication thereof are included 
in previously referred to U.S. Patent Application Nos. 09/819,922, 10/217,805, 
5 and 10/628,691. 

[0061] As shown in FIG. 4, the camera 12 is positioned near the 
composite structure 22 so as to capture images of portion of the composite 
structure being illuminated, which is typically immediately downstream of the nip 
point at which a composite tow is joined with the underlying structure. 

10 Alternatively, and as shown in FIG. 5, a reflective surface 16 may be positioned 
near the composite structure (the composite structure is not shown in FIG. 5), 
and angled such that the reflective surface 16 reflects an image of the illuminated 
portion of the composite structure. The camera 12 may be positioned to point 
toward the reflective surface 16 in order to capture close-range images of the 

15 illuminated portion of the composite structure from the reflective surface 16. More 
than one reflective surface 16 may also be utilized in further embodiments of the 
invention in which the reflective surfaces 16 cooperate in order to direct images 
of the illuminated portion of the composite structure to the camera 12. 

[0062] A wide range of cameras can be used including 

20 commercially-available cameras capable of acquiring black and white images. In 
one embodiment, the camera 12 is a television or other type of video camera 
having an image sensor (not shown) and a lens 13 through which light passes 
when the camera 12 is in operation. Other types of cameras or image sensors 
can also be used, such as an infrared-sensitive camera, a visible light camera 

25 with infrared-pass filtration, a fiber optic camera, a coaxial camera, Charge 
Coupled Device (CCD), or Complementary Metal Oxide Sensor (CMOS). The 
camera 12 can be positioned proximate the composite structure 22 on a stand 
(not shown) or mounted to a frame 28 or similar device. 

[0063] In those embodiments that do not include a reflective surface 

30 16, the camera 12 may be mounted to the frame 28 by way of a bracket 30 and 
associated connectors 32, as shown in FIG. 4. The connectors 32 may be rivets, 
screws or the like that mount the camera 12 to the frame 28 in a stationary 
position. Alternatively, the connectors 32 may be a hinge-type connector that 
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permits the camera 12, light source 14, and associated assembly to be rotated 
away from the composite structure 22. This embodiment is advantageous in 
situations where other parts of the material placement device, particularly the 
parts located behind the camera 12 and associated assembly, must be accessed, 
5 such as for maintenance, cleaning, or the like. 

[0064] FIG. 5 illustrates an alternative embodiment of the hinge-type 
connector 32 that mounts the camera 12, reflective surface 16, light source 14, 
and associated assembly (e.g., camera assembly) to the frame 28 by way of a 
bracket 30. A suitable fastener, such as a thumbscrew or any other fastener that 
10 may be removed or loosened with relative ease, can be inserted through hole 34 
and then tightened to secure the camera assembly in place for operation. The 
fastener may be loosened or removed, for example, to rotate the camera 
assembly away from the compaction roller 20 and other parts of the fiber 
placement device. 

15 [0065] With further reference to FIG. 4, a filter 15 can be placed on 

the lens 13 for filtering light in a particular manner. In one embodiment, the filter 
15 is designed to filter light such that only the infrared component or a certain 
infrared wavelength or range of wavelengths of light can pass into the camera 12. 
In this manner, the filter 15 prevents ambient visible light from entering the 

20 camera 12 and altering the appearance of the captured image. 

[0066] Other methods of filtering light can also be used to achieve 
the same, or at least similar, result. For example, the camera may be designed to 
include a built-in filter of equivalent optical characteristics. In addition, the filter 
can be located between the camera lens 13 and image sensor. Alternatively, the 

25 camera may include an image sensor that is only sensitive in the infrared 
spectrum (e.g., an infrared-sensitive camera), thus eliminating the need for the 
filter. 

[0067] The light source 14 of the system 10 will now be described in 
more detail. The light source 14 is positioned to emit light for illuminating at least 
30 a portion of the composite structure 22. 

[0068] In FIG. 4, the light source 14 is shown positioned at an 

oblique angle 37 relative to the composite structure 22. The oblique angle 37 may 
be about forty-five degrees, although other angles are possible depending on the 
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application. In addition, the light source 14 is also shown positioned to emit light 
in a direction substantially perpendicular to the direction of placement of the strips 
24 in order to highlight the defects 36, as described below. 

[0069] Further, the system 10 may include more than one light 
5 source. For example, the embodiment of FIG. 5 includes two light sources 14 
positioned relative to the composite structure and compaction roller 20 on either 
side of the reflective surface 16 and camera 12. Another exemplary embodiment 
that includes two light sources 14 is shown in FIG. 7 in which two linear optical 
fiber arrays are positioned on opposed sides of the camera 12. 

10 [0070] In FIG. 4, the light source 14 is adjustably positioned relative 

to the composite structure 22 by mounting or attaching the light source 14 to a 
mounting apparatus 27. The mounting apparatus 27 can include a main shaft 29, 
a secondary shaft 31 , and a locking clamp 33 for quickly and accurately adjusting 
the position of the light source 14. The mounting apparatus 27, in turn, can be 

15 attached to the frame 28, to the camera 12, to the bracket 30, or to some other 
object that defines a common position for both the light source 14 and the 
camera 12 such that the light source 14 and camera 12 maintain a constant 
spatial relationship relative to one another. 

[0071] The quality and magnitude of the surface illumination of the 

20 composite structure is greatly affected by ambient lighting and by the reflectivity 
of the material. Accordingly, embodiments of the invention advantageously 
employ an infrared light source to more effectively illuminate dark flaws on a dark 
background. In this regard, the light source 14 can be selected from an infrared 
light or another type of light having an infrared component, such as a halogen 

25 light source (FIG. 6) or other incandescent light sources (not shown). In other 
embodiments, the light source 14 can also include a fluorescent light source 
(e.g., white light LEDs, low pressure/mercury filled phosphor glass tube, etc.), a 
strobe or stroboscopic light source, a noble gas arc lamp (e.g., xenon arc, etc.), 
metal arc lamp (e.g., metal halide, etc.) and a lasers (e.g., pulsed lasers, solid 

30 state laser diode arrays, infrared diode laser arrays, etc.). The light from the light 
source 14 may also be pumped from through optical fibers to the point of 
delivery, such as is shown in FIG. 7. 
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[0072] In some embodiments, the light source 14 is operated at a 
power level that maximizes, or at least significantly increases, the infrared (IR) 
component of the light which works well for inspecting dark tow material, such as 
carbon. In this regard, exemplary power levels in the range of up to about one 
5 hundred fifty watts (150W) in the wavelength range of about seven hundred 
nanometers to one thousand nanometers (700nm-1000nm) have been sufficient. 
However, the particular power levels and wavelengths for the light source will 
likely depend at least in part on the camera's speed and sensitivity, speed at 
which the material is being laid, delivery losses, and reflectivity of the material 
10 being inspected, among other factors. For example, in other embodiments, 
wavelengths and power levels suitable for inspecting highly reflective materials 
can be employed. 

[0073] In the embodiment shown in FIG. 4, the light source 14 may 

comprise a plurality of LEDs arranged in an array or cluster formation. In one 
15 specific embodiment, the light source 14 includes 24 LEDs mounted in an array 
upon a three-inch square printed circuit board. 

[0074] In another embodiment shown in FIGS. 5 and 6, the light 

source 14 includes four halogen light bulbs 38, although other quantities can also 
be used. 

20 [0075] In the embodiment shown in FIG. 7, the light source 14 

includes two linear optical fiber arrays positioned on opposite sides of the camera 
12. The arrays emit light supplied from a remote source (not shown) through an 
optical fiber bundle 25. An illuminated linear array 14 is shown in FIG. 8. 

[0076] Referring back to FIG. 5, the system 10 may further include a 

25 light reflection element 18 located near the light source 14. The reflection element 
18 include a series of light reflecting surfaces 40 (FIG. 6) that redirect the light 
towards the desired area to be illuminated. This levels the illumination across the 
surface and eliminates, or at least substantially reduce, areas of intense light (i.e., 
hotspots) created by the brightest portion of the light source 14. Hotspots are 

30 undesirable because hotspots prevent consistent illumination of the composite 
structure, which may lead to errors during the processing of the images captured 
by the camera 1 2. 

15 
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[0077] The light reflection elements 40 are particularly 
advantageous for illuminating curved/contoured surfaces of composite structures 
because the redirection of the light permits a larger portion of the composite 
structure to be evenly illuminated. 
5 [0078] As shown in FIG. 6, the reflection element 18 is curved 

around the light source 14, such as in a parabolic shape. On the surface of the 
reflection element 18 that faces the light source 14, the reflection element 18 
includes curved steps 40 substantially parallel to the light source 14. The 
distance between and curvature of the steps 40 can be chosen to be sufficient to 

10 provide even illumination from the sum of the two light sources, one on either 
side of the region of interest. This enables the reflection element 18 to provide 
more consistent illumination of the composite structure 22, which prevents, or at 
least reduces, image processing errors due to inconsistent illumination of the 
composite structure 22. Alternatively, the shape and/or surface configuration of 

15 the reflection element 18 can be modified in other ways that also produce 
consistent illumination and scattering of the light produced by the light source 14 
over the desired portion of the composite structure 22. 

[0079] In an exemplary embodiment, the reflection element 18 has 
an overall parabolic shape with seventeen parabolic curved steps 40 having a 

20 range of widths from about 0.125 inches at the outer edge of the reflection 
element 18 to about 0.250 inches at the center of the reflection element 18. The 
reflection element 18 also has a uniform step height of about 0.116 inches. In 
other embodiments, however, the reflection element may be provided with 
different numbers of steps having different uniform or varying widths and different 

25 uniform or varying step heights. 

[0080] Furthermore, the reflection element 18 may be adjusted in 
order to direct the light produced by the light source 14 and scattered by the 
reflection element 18 toward the desired portion of the composite structure. For 
example, as shown in FIG. 6, the reflection element 18 is adjustably mounted to 

30 the mounting apparatus 27 with fasteners 42. The loosened fasteners 42 can 
move within slots 44 to correspondingly adjust the angle of the reflection element 
18 relative to the composite structure. Once the reflection element 18 is 
positioned appropriately, the fasteners 42 are tightened to secure the reflection 

16 
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element 18 in the desired position. Adjustments of the reflection element 18 can 
also be enabled by other methods, such as by electronic means that permit 
remote adjustment of the reflection element 18. 

[0081] It has been observed that the composite structure 22 
5 produces high glare when illuminated across the direction of placement of the 
strips 24 but produces substantially less glare when illuminated along the 
direction of placement of the strips 24. The systems and methods of at least 
some embodiments exploit the high-glare/low-glare phenomenon by casting light 
across the top layer of the composite strips 24 in a direction substantially 

10 perpendicular to the direction of placement of the strips 24. This produces a 
relatively large amount of glare on the top layer of the composite structure 22. 
The underlying layers, which produce significantly less glare than the top layer 
because of their orientation, will show through any gaps or other defects in the 
top layer and thus be easily located. In addition, twists and other surface defects 

15 in the top layer will alter the orientation of the strips in the top layer and thus 
correspondingly alter, i.e., decrease, the glare of the top layer at the defect 
location. 

[0082] While the high-glare/low-glare phenomenon occurs when 
illuminated with either visible light or infrared light, the filter 15 used in one 

20 embodiment of the system 10 substantially removes the glare caused by ambient 
light such that only the glare caused by the infrared light source is used to locate 
the defects. Accordingly, the filter 15 removes the interference of ambient light as 
the composite structure 22 is being examined for defects. 

[0083] In any of the system embodiments described herein, there 

25 may be one or more cameras 12 and/or one or more light sources 14 with or 
without reflection elements 18 (collectively referred to as light sources, 
hereinafter). In addition, the one or more cameras 12 and/or the one or more light 
sources 14 may be moveable relative to the composite structure. The multiple 
cameras 12 and/or multiple light sources 14 and the moveability of the camera(s) 

30 12 and/or the light source(s) provides system 10 flexibility in order to capture the 
most accurate images of the composite structure. Multiple and/or moveable light 
source(s) 14 permit consistent and sufficient illumination of the desired portion of 
the composite structure, regardless of the shape of the composite structure. 
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Likewise, multiple and/or moveable camera(s) 12 enable capturing an accurate 
image of any area of the composite structure, regardless of the shape of the 
composite structure. As such, the multiple and/or moveable light source(s) and/or 
camera(s) are particularly advantageous when illuminating and capturing images 
5 of and curved/contoured portions of composite structures. The multiple and/or 
moveable light source(s) and/or camera(s) are also advantageous in illuminating 
and capturing images of composite strips having a width that makes it difficult to 
illuminate and/or capture images of the entire strip, such that the position of the 
light source(s) and/or camera(s) may be moved over the entire strip, and/or 
10 multiple stationary light source(s) and/or camera(s) may be positioned to cover 
the entire strip. Systems including moveable cameras and light sources are 
described in detail in previously referred to U.S. Patent Application No. 
10/217,805. 

[0084] As shown in FIG. 4, the system 10 can also include a 

15 marking device 62 for marking the location of defects on the composite structure 
22. The marking device 62 may be attached to the frame 28 and be triggered by 
a processor 66 or similar device when a defect 36 is detected. The marking 
device 62 may spray or otherwise deposit an amount of ink, paint or the like onto 
the composite structure 22 in those areas where defects 36 has been detected. 

20 The markings on the composite structure 22 enables the location of the defects to 
be subsequently readily identified either automatically or manually. 

[0085] In the particular illustrated embodiment, the marking device 
62 is an inkjet marking system that sprays a small spot of compatible ink of a 
highly visible color onto the surface of the composite structure 22 at the defect 

25 location to permit rapid access for repair and disposition. Alternatively, other 
marking methods can also be used, such as a pump-fed felt-tip marker, spring- 
loaded marking pen, audio or visual alerts, and the like. 

[0086] The camera 12 and/or the reflective surface 16, which along 
with the light source 14 and any reflection element 18, can be mounted to the 

30 head unit to allow the camera 12 to continuously capture real-time images of the 
composite structure 22 and the strips 24 as the head unit moves across the 
composite structure 22 and the composite strips 24 are laid down. If the 
composite structure 22 is not planar, the inspection point should preferably be as 
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close to the nip point as possible, as described above. If the composite structure 
22 is planar, the inspection point can be located further from the placement head 
unit. In either case, the images can be stored in a memory device 64 for future 
analysis and/or processed immediately by the processor 66, as discussed more 
5 fully below. 

[0087] FIG. 9 shows an exemplary raw or unprocessed camera 
image 68 illustrating a contrast between potential defects, such as a pucker 75 
and a twist 77, and the remaining portions of the composite structure 22 that are 
defect free. In the illustrated embodiment, the potential defects 75 and 77 are 
10 shown as black or gray areas, while the remaining non-defective portions of the 
composite structure 22 remain substantially white 72. Once the potential defects 
are located, the defects may be marked with the marker 62 and the linear and 
lateral distances to the potential defects can be determined in a manner describe 
above. 

15 [0088] With further reference to FIG. 4, the processor 66 may 

receive the images 68 from the camera 12 or from the memory device 64 in 
which the images 68 have been stored. The processor 66 may then process and 
analyze the images to facilitate the reliable detection of defects. In at least one 
embodiment, the processor 66 and memory device 64 are components of a 

20 conventional computer. 

[0089] The system 10 may also include a user interface 76 that is in 
communication with the processor 66. The user interface can be programmed 
such that it can run from a wide range of software applications, including but not 
limited to DOS, Windows 98, Windows/NT, Windows 2000, Windows CE, Linux, 

25 Unix, and equivalents. 

[0090] As shown in FIG. 10, the user interface 76 includes a display 
screen 80, such as on a computer monitor, and can also include an input device, 
such as a keyboard and mouse (not shown), for permitting an operator to move a 
cursor about the display screen 80 and input various system settings and 

30 parameters. The display screen 80 can also be touch-sensitive for permitting the 
operator to input the desired settings by manually touching regions of the display 
screen. 

19 
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[0091] The user interface 76 includes a window 81 in which an 
image 74 of the composite structure 22 is displayed for viewing by the operator or 
other user. The window 81 can also include a visual display 69 of defect location 
by course. 

5 [0092] Although the image 74 can be the unprocessed camera 
image 68 (FIG. 9), the image 74 shown in FIG. 10 can also be a processed 
image that has been binarized. During binarization, all shades of gray above a 
predetermined threshold value can be changed to white, while all gray shades 
below the threshold are changed to black to heighten the contrast of defects and 

10 improve the accuracy of defect detection. In other embodiments, the binarization 
operation need not be performed but instead the raw image, rates of change of 
the light levels in the raw image, and/or color changes in the images can be used 
to identify the defects. 

[0093] The user interface 76 also provides user controls 78 for 

15 allowing various user inputs to the system. In the particular illustrated 
embodiment of FIG. 10, the user interface 76 allows adjustment to the 
binarization threshold. Generally, the setting of the binarization threshold involves 
a tradeoff between the sensitivity with which defects are detected and the 
resolution with which the defects are depicted. In one embodiment, the 

20 binarization threshold is set to about 128 which corresponds to the mid-point on 
the 8-bit digitizing range of 0 to 255. However, other binarization threshold values 
can be employed depending at least in part on the particular application, 
available lighting, camera settings, among other factors. 

[0094] The user controls 78 also allow the user to adjust or shift the 

25 viewing area within the window 81. During operation, the window 81 displays 
real-time moving video images of the illuminated portion of the composite 
structure 22 as the camera 12 and/or the reflective surface 18 are moved relative 
to the composite structure 22. 

[0095] The interface 76 can also allow the user to input the width of 

30 course or tow band 71 and maximum allowable cumulative gap width 73. 

[0096] In addition to displaying images of the composite structure 
22, the display screen 80 also includes a defect table 82 which lists the 

20 
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discovered defects and provides information for each defect, such as location, 
size, and the like. 

[0097] The display screen 80 can also provide information (which 
can be continuously updated) such as the number of defects 50, number of 
5 courses completed 52 (which may be determined by counting pressure on/off 
signals from the machine load cell as described above), cumulative defect width 
54, and length of the current defect being measured 56. 

[0098] The display screen 80 can further include status indicators 
84 that notify the user whether a particular image area is acceptable or not 
10 acceptable based on predefined criteria, such as maximum allowable 
dimensional parameters and tolerances. 

[0099] The display screen can also include an indicator 85 that 
notifies the user when the allowable cumulative defect width limit has been 
exceeded. 

15 [0100] An exemplary embodiment includes importing a part model 

from external or third party software (e.g., computer aided drafting (CAD) 
programs, work station-based programs such as Unigraphics (UG) or CATEA, 
desktop PC applications such as AutoCAD, etc.) 

[0101] FIG. 11 illustrates an example of a complex part model 90 

20 imported from third party software. As show in FIG. 12, a course grid overlay 92 
can be constructed for the imported part model 90 using the number of courses 
and the direction of travel that correspond to ply orientation. The diagrammatic 
concepts of linear and lateral distances 19 and 21 are illustrated in FIG. 12 to 
show defect 36 location on a surface more complex than the surface shown in 

25 FIG. 1. 

[0102] After all courses for a ply have been laid, the course grid 
overlay 92' is repositioned to represent the change in orientation or direction of 
travel for the new ply, as shown in Figure 13. The interaction between the 
external or third party software and the software of the material placement 
30 machine can be designed to generate an entire set of grids (one for each ply of 
the part) in advance. These grids can be stored, with the appropriate grid being 
accessed, called up and positioned at the start of each ply. 
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[0103] FIG. 14 illustrates another embodiment in which two 
computer displays are employed for displaying and tracking the various defect 
data for the imported model 90. As shown, one monitor displays the computer 
display 80 (previously described above in reference to FIG. 10) while the other 
5 monitor simultaneously displays a computer display 1 80 of the part model 90 and 
course grid overlay 92. The computer displays 80 and 180 can be continuously 
updated to show positioning and locations of defects and flaws as they are 
detected through the vision system interface, described above. 

[0104] FIG. 15 illustrates another embodiment 280 in which the 

10 display 180' of the part model 90 and course grid overlay 92 are displayed within 
the window 81 ' of computer display 80\ 

[0105] FIG. 16 illustrates another embodiment 380 in which the 
display 80" is superimposed or positioned over a corner of the display 180" 
including the part model 90 and course grid overlay 92. 

15 [0106] Accordingly, embodiments of the present invention provide 

in-process vision-based inspection systems and methods capable of accurately 
and efficiently determining various defect characteristics such as total defect 
count, total defect width, defect density-per-unit area, cumulative defect width- 
per-unit area and/or defect locations. Embodiments of the invention allow 

20 composite structures to be fabricated more efficiently with fewer interruptions 
than conventional material placement systems which require manual inspections 
for and measuring of defects. 

[0107] Embodiments of the invention permit rapid detection and 
measurement of the cumulative defect width-per-unit area, and tracking of the 

25 defect density-per-unit area. Because this defect information is relatively 
immediately available and manual measurement is not necessary, machine 
down-time can be significantly reduced resulting in reduced manufacturing costs 
and cycle times. 

[0108] In addition, embodiments of the present invention allow for 
30 ready identification of those composite structures that exceed maximum 
allowable tolerances pertaining to density and cumulative width of defects. This 
allows the fabrication process to be halted when maximum allowable tolerances 
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are exceeded, thereby saving time and materials which would otherwise be lost 
during continued fabrication of an unacceptable composite structure. 

[0109] Further, when too many composite structures are being 
rejected, an operator can adjust the machines accordingly such that less material 
5 is wasted, less labor is expended, and less machine down time is incurred during 
the fabrication process. Therefore, a lower cost composite structure can be 
achieved on average. 

[0110] Additionally, embodiments also enable improvements in the 
overall quality of the parts produced because defect density and cumulative 
10 defect width can be determined more uniformly and reliably with the various 
systems and methods of the invention than with manual inspections. 

[0111] While various preferred embodiments have been described, 
those skilled in the art will recognize modifications or variations which might be 
made without departing from the inventive concept. The examples illustrate the 
1 5 invention and are not intended to limit it. Therefore, the description and claims 
should be interpreted liberally with only such limitation as is necessary in view of 
the pertinent prior art. 
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